Skip to content

Split native and Panama OSQ vector scorer implementations#144667

Open
ChrisHegarty wants to merge 8 commits intoelastic:mainfrom
ChrisHegarty:split-native-panama-osq-scorers
Open

Split native and Panama OSQ vector scorer implementations#144667
ChrisHegarty wants to merge 8 commits intoelastic:mainfrom
ChrisHegarty:split-native-panama-osq-scorers

Conversation

@ChrisHegarty
Copy link
Contributor

@ChrisHegarty ChrisHegarty commented Mar 20, 2026

The OSQ vector scorer methods have grown cluttered with nested if/else branches testing NATIVE_SUPPORTED, SUPPORTS_HEAP_SEGMENTS, and VECTOR_BITSIZE , all static final constants. I am concerned about maintenance and JIT inlining, since the large bytecode size of these methods could prevent the JVM from inlining them on the hot path. So I split the native (FFI) and Panama (Vector API) implementations into separate class hierarchies.

A NativeMemorySegmentScorer abstract base provides template methods for quantize, bulk scoring, and corrections. Three small final subclasses (one per quant type) supply only the native dot-product function pointers and bit scales. The existing MS* scorers are now Panama-only, with all native branches removed. The right implementation is selected once at class-load time via a static factory map, keyed on query/index bit combination.

I also moved the dataLength >= 16 SIMD minimum check from every method call to construction time, so the native hot-path methods have no unnecessary runtime guards. A nice consequence of this is that the filtered KNN path (scoreBulkOffsets) now uses fully native scoring, both quantize and bulk corrections end-to-end. Previously it used native quantize but fell back to scalar corrections. I also replaced readInt() loops with bulk readInts() in the base ESNextOSQVectorsScorer.

@ChrisHegarty ChrisHegarty added >refactoring :Search Relevance/Vectors Vector search test-arm Pull Requests that should be tested against arm agents Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch labels Mar 20, 2026
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

@ChrisHegarty
Copy link
Contributor Author

Buildkite benchmark this with so-vector please

@ChrisHegarty
Copy link
Contributor Author

Buildkite benchmark this with dense-vector please

@ChrisHegarty
Copy link
Contributor Author

Buildkite benchmark this with so-vector please

@ChrisHegarty
Copy link
Contributor Author

Buildkite benchmark this with dense-vector please

@elasticmachine
Copy link
Collaborator

elasticmachine commented Mar 22, 2026

💔 Build Failed

Failed CI Steps

This build attempts two dense-vector benchmarks to evaluate performance impact of this PR. To estimate benchmark completion time inspect previous nightly runs here.

History

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>refactoring :Search Relevance/Vectors Vector search Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch test-arm Pull Requests that should be tested against arm agents v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants